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ABSTRACT 

A review of fiscal and political trends in the late 
1970 's made it appear that support for education would slip 
measurably, but a resurgence of national interest in education in the 
early 1980 's as a result of commission reports and nationwide studies 
has led to increased spending for major reforms. The pattern of 
support for reforms in other areas suggests that the public must be 
convinced that reforms are having an impact if their support is to 
continue. This need to convince the public can be translated into a 
need for effective means of measuring the results of the educational 
reform efforts. Among the methods that can be used for evaluating the 
success of reforms are (1) monitoring statewide and local performance 
indicators, (2) studying the overall financial impact of the reforms, 
(3) comparing the cost effectiveness of various programs with similar 
goals, (4) conducting evaluations of discrete programs, (5) assessing 
the degree of local implementation of state-mandated reforms, (6) 
examining the cumulative impact of broad reform packages, and (7) 
conducting research isolating cause and effect relationships. Such 
evaluations are extremely complex, but are critical to maintaining 
reform momentum. (PGD) 
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The research activity of the institute is divided into the 
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Organizations. In addition, there are a number of other projects and 
programs in the finance and governance area that are sponsored by 
private foundations and government agencies which are outside of the 
special R&D Center relationship with NIE. 
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Abstract 



This paper analyzes the recent trends in state education finance with 
particular emphasis on the role of reform in overcoming negative systemic 
trends. The major thesis is that the recent increases in state fund? can only 
be sustained if there is a favorable perception of recent reforms. The final 
section presents seven different approaches for thinking about assessing the 
recent state reforms. States need to conceptualize and execute some of these 
seven very quickly. Assessment is used here as a much broader concept than 
testing. 
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The Emerging Context of the 1980 s 

Public school policymaking is embedded in a societal matrix of numerous 
forces. It is not possible to consider the future of our schools without 
examining the size and distribution of future populations, the future of the 
economy and its effect on money available for schools, and the political 
context within which decisions will be made. The public school system is a 
"dependent variable" of larger social and economic forces. These forces 
are sometimes cyclical in nature.^ For example, in the late 1950s, launch- 
ing Sputnik triggered a series of policies that directed resources toward 
the training of gifted students, especially in science. In the mid-1960s. 
President Johnson's War on Poverty produced counter-trend poli cites through 
the redirection of resources to the disadvantaged and the handicapped. In 
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the 1980s, the concern about the U.S. competitive economic position focussed 
public policy on academic standards and quality for all pupils. 

Educational policies are also often determined by the actions of 
special interest groups external to the education system. California's 
Proposition 13 and other state spending caps, for example, resulted from 
taxpayer resentment against the property tax and inflation. Schools were 
affected deeply by an issue in which they were only an indirect target. 
Recently, special interest groups directly related to education have had a 
less powerful effect on the major directions of state education policy. 
Their policy influence has been muffled not only by broad social forces 
and taxpayer groups, but by external authorities who are more distantly 
connected with education policy such as governors, business leaders, and 
courts. 

In 1979, Kirst and Garms made predictions about the demographic, fiscal, 
and political context of public education for 1980-1990. We predicted that 
education expenditures would keep up with inflation, but not result in con- 
siderable real growth. This was a more pessimistic scenario than the 
prior 1970-1980 decade. Despite considerable discussion about declining 
enrollment and lower public approval ratings in the Gallup Poll, the 1970s 
produced a $23 billion growth in total expenditures after inflation. 

The ratio of pupils to instructional personnel fell from 29 to 1 to 
under 20 to 1 by the dawn of the 1980s. A cot.tinuation of this trend would 
result in a 12 to 1 ratio during the 1990s! State governments fueled this 
1970-1980 expenditure growth by increasing their total education expenditure 
from $16.6 billion to $46.5 billion, an impressive 44.5 percent increase in 
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' real terms. The state share rose from 37 percent to nearly 50 percent, while 

the local and federal shares declined. All of these positive fiscal trends 
occurred despite the "management of decline" being featured in professional 
education journals. School closings and enrollment decline were indeed pain- 
ful, but hold-harmless fiscal distribution formulas cushioned the impact in 
most states. There was a continued trend toward state policy initiative and 
a narrowing of the discretionary decision zone of the local school authorities 
as states stressed an equity agenda for the handicapped and disadvantaged. 

Looking ahead to the 1980s, Kirst and Garms were not optimistic that 
the 1970s expenditure growth could continue at the same pace. They discerned 

•nany negative societal trends, including: 

Demography - enrollment would drop in the high schools, but more 

important was the rapid growth of older voters who tend to want lower 

• property taxes. The numbar of voters with children in public schools 

would be about one in every five. The fastest growing school populations 

were immigrants who have a low political participation rate. All of these 

factors would depress local voter approval of property tax increases. 

Energy Prices - oil prices began declining and this would hurt the 
Southern and Western states that rely on extraction taxes. 

Public Alienation from Scho ols - the Gallup Polls demonstrated a grow- 
ing dissatisfaction with school performance that might translate into less 
political support. 

Growing Competition from Child Care Meeds - the labor force participa- 
tion rate for women with school age children had increased so rapidly that 
between 70 to 80 percent of the mothers would be working by 1990. Moreover, 
there was a rapid growth in divorced or single parent families. 

o 6 
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Federal Budget Priorities - federal policy favored defense, social 
security, and health programs. Total expenditures were Increasingly going 
to older people rather than children. 

Kirst and Garms felt the state governnients would be the engine for real growth 
in school spending. Federal budget deficits and priorities were moving away 
from education and the changing profile of local voters would inhibit local 
property tax increases. 

Public School Policy 1980-1985 

In the early 1980s, the pessimistic view prevailed as education's real 
t^^tal revenues (after inflation) declined from 1980 to 1982. The U.S. 
suffered a recession that devaftated many of its basic industries. But in 
1983 the fiscal and political picture for education changed dramatically and 
unexpectedly. Education became the top priority in most states as a wave of 
concern about "academic excellence" swept the nation. The underlying negative 
trends highlighted in the prior section were overwhelmed by the public willing- 
ness to fund quality "reforms." Education was featured as a solution to 
economic stagnation and international competition. About 300 state commissions 
and many more local groups pushed for a new education agenda. Expenditures 

per pupil shot up about 9 percent in real terms in 1983 and kept increasing 

3 

faster than inflation in 1984 and 1985. 

In my view, the crucial policy question for the next five years is 
whether this "reform momentum" will keep going. If it does, then education 
expenditures will exceed inflation and the underlying negative trends pushed 
to the background. If the public and key policymakers, however, perceive 
that education reform has failed or not been implemented, then a less opti- 
mistic future is likely. The public must not perceive that professional edu- 
cators have subverted the aims of the reformers. Consequently, implementation 



and assessment of these reforms should be a top priority for educational 
research and state po'iicymakers. By 1986, the pace of new reforms will slow 
significantly as fewer states and LEAs enact new omnibus tills or interven- 
tions. Moreover, national economic growth is slowing dramatically, setting 
up tough competition for public funds. Researching the- cost/ 
effectiveness of the varied education reforms becomes urgent, because not 
all of them can be expanded or even maintained. 

Education policy is now beyond the "alarmed discovery" and "crisis 
activity" phases of 1983-1984 of the issue-attention cycle. Other reforms 
like ecology and urban development have degenerated into the subsequent 
policy phases of "disillusionment with results" and then a "return to 
neglect."^ The process of implementation, adaptation, refinement, and 
elimination of the state reforms has begun in earnest. For example, there 
is a widespread realization that teacher quality is crucial to increased 
academic attainment, but states are unsure how to blend screens and magr,>its 
to attract and retain high quality teachers. Consequently, states are 
trying all kinds of teacher quality interventions, including career ladders, 
minimum salaries, mini grants, sabbaticals, and forgiveable loans, without 
a clear notion of which approaches are optimal. I doubt that many states 
can afford to fully fund all of the various schemes to improve the teaching 
profession. 

Sustaining the reform momentum becomes even more urgent when one con- 
siders that national enrollments will grow by 2.1 million by 1990. Allan Odden 
estimates that it will take 5 percent annual real growth in total revenues 

merely to pay for this enrollment increase! He emphasizes that total 
revenue grew only 7 percent between 1980-1985 (per pupil revenues were up 



18 percent because of the reform nx)nientum, resiliency of the property tax, 
and state hold-harmless formulas that do net cut spending commensurate with 
pupil decline). The reforms recommended by the National Commission on 
Educational Excellence would require about a 20 percent increase in per 
pupil expenditures. Only two out of eight reform states that Odden studied 
have approached this level of increase.^ Since increases of this magnitude 
are unlikely, we must sort out which of the many state reforms should be 
expanded, eliminated, or left at their current funding level. This task 
will becoffje more urgent if the President's tax reform bill passes that would 
end the deductibility of state and local taxes from federal income tax. This 
deduction elimination will mak? it harder to increase state and local taxes. 

Evaluating Education Reform: An Urgent Priority 

While the reform movement has been a major positive force for state 
education policy priority, politicians are clamoring for results. There 
are numerous unanswered questions. How does one assess omnibus bills like 
California's SB 8'i3 with its 80 different "reforms"? Evaluators have 
focussed on program evaluation, but these state reform packages are not 
programs. They are a welter of specific state interventions aimed at curri- 
culum and instruction where evaluation methoojiugy is not well developed. 

There are many urgent questions subsumed within these state omnibus 
reforms. What is the proper balance between state and local policy control? 
Will bottom-up commitment at the school site reinforce top-down state leader- 
ship and lead to a partnership toward the same goals? While most LEAs are 
increasing required academic courses, what will make all students motivated 
and interested in these courses? Are there some reforms like merit pay that 
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outrun our present technology? As science and math enrollments ir.crease 

dramatically (about 20 percent between 1982-1984 in California high schools), 

who will teach these courses? What will happen to the dropout rate? 

In 1985 the 50 states provide an unprecedented natural setting to answer 

these and other questions. The major policy issues of the next five years 

will focus on thesf state refonn problems, successes, and unanswered 

questions. We need to involve scholars and practitioners from a variety of 

perspectives to help answer quickly numerous .omplex questions. Fortunately, 

there is such diversity in the state policy approaches and local contexts 

that we have a quasi -natural experiment across the nation. For example, 

some states have imposed a statewide career ladder model, while othe^^s have 

relied on locally generated differences in approaching career structures. 
An evaluation of all the policy issues raised in the 1983-1985 state reforms 

will be very expensive, but only a few states such as South Carolina, Florida, 

and Tennessee have earmarked significant money for in-depth analysis of the 

impact or success of these reforms. It is ironic that education has so 

much at stake on the public perception of these state reforms, but is devoting 

so few resources to assessing their OtJtcomes, 

Spec'ific Evaluati o n Approaches fcr State Refonri 

There are several types and levels of evaluation required for the com- 
plex and multi-purpose state reform bills, 

1 . Performance indicators or statewide barometers of impact . Perform- 
ance Indicators are statewide numerical measures of trends in educational 
variables. In some cases these standardized state measures are supplemented 
by locally devised indicators that vary in definition and concept depending 
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on local conditions. These performance indicators pick up easily measured 
changes but can rarely penetrate behind the classroom door to include con- 
tent taught, teacher morale, or the type of intellectual tasks students are 
learning. They are part of a statewide strategy, but if used alone tend to 
overemphasize what can be measured at the state level. Bill Honig in 
California has created statewide targets and individual school profiles for 
numerous uniform indicators, including changes in course-taking patterns, 
performance in the freshman year of college, tests, and dropouts. These 
state indicators are supplemented by locally devised indicators in such 
areas as school climate, time spent on writing, and homework that are not . 
easily collected at the state level. 

2. Overall studies of the financial impact of the reforms . Rather than 
cost out each refomi carefully, states backed into the amount allocated to 
LEAs based on uncommitted state revenue. Often money was provided th»*ough 
the state's basic finance formula and not tied to specific reforms. Conse- 
quently, states need to know where the LEAs spent the increased money and 
if some reform components are over- or under-funded. For example, how much 
was needed to institute state required science classes, or create new ap- 
proaches for local teacher aval uation? What ar .s of the curriculum gained 
more money (probably math and science), and which areas lost money (indus- 
trial arts and home economics are likely losers)? As reforms became Dart 
of collective bargaining, did they become more or less expensive than state 
planners expected? Did the local use of money vary according to the assessed 
value or prior spending level of the LEA? 

In order for a reform to have any chance of impact, some new resources 
usually must be provided. Cost analysis will help answer this question, but will 
not tell us anything about results. 
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3. Cost/effectiveness analysis of various state interventions with 
the same specific objectives > Levin states the case of cost-effectiveness 
this way: 

...it integrates the results of their co^c«" in such a way that 
one can select the best educational result.- for any given costs, 
or that provide any given level of educational results for least 
cost. It is important to emphasize that both the cost and effec- 
tiveness aspects are important and must be Integrated. Just as 
evaluators often consider only the effects of a particular 
alternative or intervention^ administrators sometimes consider 
only costs. In both cases, the evaluation will be incomplete ' 
(p. 15). 

Cost-effectiveness can provide important policy information, but it is 
limited to comparisons among programs witn similar objectives. A possible 
example would be the use of loans, scholarships, or a higher minimum salary 
as a magnet to attract better quality beginning teachers. 

4. Program evaluation . Some of the states created discrete programs 
^ that can be evaluated as discrete activities. Some examples would be: 

teacher career ladders, increases in high school counselors, or pre-school 
programs like Headstart. Programs that were implemented can be evaluated 
using the well -developed techniques of program evaluation that have assessed 
such programs as Title I of the federal Elementary and Second? -y Education 
Act.S As a first step, program evaluation can research what components of 
a program were actually implemented. It can then move to include costs, 
outcomes, and processes. A comparison of several programs with very similar 
objectives wou^d be called cost-effectiveness. 

Program evaluation can not address well the interaction effects among 
several different state initiatives. Nor can it give us much insight on 
the aggregate and cumulative impact of omnibus state reform activities. 

5. Impact of evaluation of several state interventions with the same 
general goal. As of 1985, It is premature to ask whether state reform is 

er|c 1 i 
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working in terms of student achievement. It is sensible, however, to 
explore whether or not a reform is implemented, and if so, whether this 
Implementation is consistent with the broad objectives of state policymakers. 
Numerous state policies are directed toward a similar goal of increasing 
the "rigor" or "challenge" in the high school curriculum. The primary strat- 
egy Is to Increase student time in traditional academic subjects. Minimum 
state graduation standards, elevated college entrance requirements, model 
state curriculum, and the addition of science and math sections to statewide 
tests are some examples. 

An example of this type of evaluation is a recent California study that 
examines changes in high school course offerings between 1982 and 1985. A 
sample of secondary schools was surveyed. Numbers of class sections in each 
departmental area were taken from teachers' master schedules. After adjust- 
ments for changes in enrollments, it is clear that substantially more sec- 
tions of mathematics, science, and fo»'eign language courses are being 
offered while courses in home economics, industrial arts, and business are 
decreasing (Figure 1). In science, the largest increase occurred in physical 
science offerings, apparently in response to the new graduation requirement 
of one year of physical science. All areas of mathematics displayed in- 
creases, but computer science showed the largest increase (+91 percent) 
followed by more advanced math courses such as calculus, analytic geometry, 
trignometry, and geometry. Calculus and analytic geometry courses were 

offered with 33 percent greater frequency while general math courses 
Increased by only 11 percent. 

The same study showed an even more substantial increase in the number 
of advanced placement course offerings. Additions to courses which had been 
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Figure 1. 



Fttrcttnt Chang* in Mumb^r of Course Sactions Off«r«d, 
Adjusted for Changes in Inrollment, 
1982-83 to 1984-85 
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Source: Pan Grossman, Michael W. Kirst, Worku Negash, Oackie Schmidt- 
Posner, Study of Curricular Change in California Comprehensive 
High Schools; 198Z--83 to 1984-8b. Policy Paper No. PPHb-'Z-A 
I Berkeley; Honey Analysis for California Education, July 1985), 
Course sections were cross-checked with pupil course enrollment 
data derived from state reports. 
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offered in chemistry, physics, and European history) constituted, by 1984- 
85, a 34 percent increase. In addition, new advanced placement courses were 
offered in computer science, foreign languages, art history, and music. An 
obvious problem with these simple impact analyses is that there is no prp- 
tense or method to demonstrate cause and effect relationships. For instance, 
changes could be caused by local school board policies, state interventions, 
of other factors. But if the direction of change is toward more academic 
course work, state policymakers will be interested, even without a precise 

analysis of the unique state role or of the content covered within these course 
titles. 

6. Cumulative effects studies of the impact of all the reforms in state 
omnibus bills . The totality of all the initiatives in many states could 
not be conceptualized as a discrete program like Headstart or a "policy" like 
a civil rights mandate. Rather, reforms in South Carolina and Texas, for 
example, contain: 

0 Broad, multiple targets . Reform packages seek to alter simultan- 
eously several components of schools and district operations. 

0 Lack of programmatic connections . Though varying across states, 

several of these reform efforts combine under a single statute many kinds 

of reform elements. Education reform is a set of minimally- related policies 

that will generate various forms of response at the local level. 

Given the nature of these state reforms, Knapp and Stearns stress that the 

evaluator should study the local system, not the program. Specifically, 

they contend that school effects will: 

derive from many small changes that cumulatively shift the climate 
for education, the perceived opportunities, the tenor of the curri- 
culum. These shifts will be best detected by "taking the pulse" of 
the local educational system In ways that capture more than con- 
ventional Indicators such as student test scores.... 
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The many pieces of the reform agenda compete with a buzzing 
universe at the school and district levels for the attention of 
educators. Collectively, the reforms will have their greatest 
impact if they: first, capture the attention of a critical mass 
of educators (and their relevant local constituencies); second, 
provide positive incentives for committing further energy to 
education (by current staff, as well as by new recruits); and 
third, generate hope for, and supportive imagery of, schools 
among students, educators, and the public. Accordingly, 
evaliation research must document what is (and isn't) noticed at 
the local level, and determine the effects reform initiatives 
have on local motivation and morale (at the administrative, teacher, 
and student levels). In such reform movements the whole is greater, 
and far more important, than the parts. Those aspects of the local 
scene that refect the whole— such as the commitment educators 
feel to reform joalS"are-,consequently the most appropriate indi- 
cators of reform effects. 

In short, local case studies and state level interviews would examine commit- 
ment of local actors to state reform objectives, adequacy of rerources to 
support local efforts, and the implementation structure. 

In doing this kind of evaluation, it is essential to differentiate 
initial from long-run Impacts, and be cognizant of changes in local response 
over time. The initial effects will be on inputs and processes such as 
changes in course-taking patterns of students. Only after several years 
should we expect significant changes in standardized tests. This type of 
research is extremely complex, expensive, and takes a long time before even 
initial results are reported. It requires longitudinal, in-depth case studies 
of a sample of local schools within each state. 

7. Re search that Isolates the cause and effect relationships . While 
1-6 above will be very useful, state policymakers need to know whether there 
are Iden'cifiable cause-effect relationships between student achievement out- 
comes and Interventions like curriculum alignment. This will be difficult 
and expensive. How can we separate out local policies from state policies 
if both are reinforcing each other? Moreover, some of the state policies 



18 



-14- 



may conflict and contradict each other. For instance, magnets such as career 
ladders may attract better teachers, while state-mandated, test-driven curri- 
culum standards repel them. Cause and effect reseairch probably is best 
attempted in the latter stages of state assessment of reform. We should dis- 
cover if a program has any impact or has even been implemented before under- 
taking sophisticated cause and effect studies. There is no sense in research- 
ing cause and effect if a program never attracted the attention of local 
educators. 

Some experts contend that cause and effect or input/output studies are 
not very appropriate for state- reforms. The effects of social programs cannot 
be proven like a principle of physics. The best use of evaluation for omnibus 
state reforms would assess implementation in varied states and local schools. 
This implementation focus is crucial because what is delivered to children from 
state reforms varies greatly by setting (e.g. schools and classrooms). More- 
over, implementation is a multi-stage developmental process whereby localities 
learn and adjust as they install reforms. State policymakers must keep in 
mind that the decisions made closest to children are the most important. Class- 
room teachers display enormous differences in how they react and adjust to 
external state reforms. 



-15- 



Conclusion 

Most of the states will have enacted the bulk of their reform efforts 
by January 1986. The ensuing years will be occupied with sifting and sorting 
impacts and promising approaches. Hopefully, state reform will not be 
judged by a single misleading indicator like SAT scores. But policymakers 
must have varied and specific evaluations ^n order to forestall such 
simplistic judgement. Almost no state has devoted the resources necessary to 
mount such complex evaluations. Yet, the reform momentum and future education 
funding are linked to public perceptions about the "success" of reform. Pub- 
lic disillusionment with this reform era could result in a renewed interest 
in public funding for private schools. 



i 



20 



FOOTNOTES 

1. For an elaboration of this point and several other issues in this 
initial section, see Michael W. Kirst and Walter I. Garms, "The Politi- 
cal Environment of School Finance Policy in the 1980s," in James W. 
Guthrie (ed.). School Finance Policies and Prac tices (Cambridge: 

Bal linger, 1980), pp. 47-78. — ^ 

2. Ibid , p. 65. 

3. See Allan Odden, "Education Finance 1985: A Rising Tide or Steady 
Fiscal State," an address to the annual meeting of the American Educa- 
tion Finance Association, Phoenix, April 1985, p. 15. 

4. Anthony Downs, "Up and Down with Ecoloqy: The Issue Attention Cycle," 
The Public Interest . 1972, pp. 39-50, No. 29. 

5. Odden, o£. cit .. p. 25. 

6. See Allan Odden, "Financing Educational Excellence," Phi Delta Kappan . 
Vol. 65, No. 5, January 1984. The two states were South Carolina and 
Tennessee as cited in Odden 's 1985 AEFA paper. 

7. Henry M. Levin, Cost Effectiveness :__ A Primer (Beverly Hills: Sage, 
1983), p.. 15. ~ ' 

r 

8. Lee Cronbach, et al.. Toward Reform of Program Evaluation (San Francisco: 
.Inssey Bass, IQBnjr 

9. Study conducted by Policy Analysis for California Education by Pam Gross- 
man, Michael Kirst, Jackie Posner, and Worku Negash, 1985. 

10. Michael Knapp and Marian Stearns, "Improving System Wide Performance: 
Evaluation Research and the State Education Reform Movement," in J. 

ul ^^f^'l* Organizational Excellence; Roles for Evaluators 
(Lexington, MA: Lexington Books, forthcoming). Entire cumulative effe'cts 
section of this article based on above paper. 

11. Milbrey W. McLaughlin, "Implementation Realities and Evaluation Design," 
(Stanford: IFG, Program Report No. 84-Bl, 1984). 



ERIC 



21 



